Micro-expression is a facial movement that humans make when they are trying to hide their true emotions. It has the typical characteristics of short duration and small amplitude. Concerning the problems of the difficulty in recognition and the unsatisfactory recognition effect of micro-expression, a micro-expression recognition algorithm based on Convolutional Block Attention Module (CBAM) and Dual Path Networks (DPN), namely CBAM-DPN, was proposed. Firstly, data fusion of typical micro-expression datasets was performed. Then, the change values of pixels in the sequence frames were analyzed to determine the position of the apex frame, after that, image enhancement was performed to the apex frame. Finally, based on the CBAM-DPN network, the features of the enhanced micro-expression apex frame was effectively extracted, and a classifier was constructed to recognize the micro-expression. The Unweighted F1-score (UF1) and Unweighted Average Recall (UAR) of the model after optimization can reach 0.720 3 and 0.729 3 respectively, which are improved by 0.048 9 and 0.037 9 respectively compared with those of the DPN model, and are improved by 0.068 3 and 0.078 7 respectively compared with those of the CapsuleNet model. Experimental results show that the CBAM-DPN algorithm combined with the advantages of CBAM and DPN can enhance the information extraction ability of small features, and effectively improve the performance of micro-expression recognition.